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. . "Acoustic echo and noise cancellation." 

Pidd of TV £A</W-tov- 



The invention relates to a filter as described in the preamble of Claim 1 . 
1^^"^ The-m vontion further relates to a signal piuc c ssiiig device comprising such a filte r. 
Tho inventi o n further relates to aixJcc u iifciciiciiig s ystem. 
. F urther the invention relates to a voice controlled electronic de vice. 
5 The invention also relates to a noise cancellation system. 
lN!^~^ ^ e i nvent i° n further relates to a method a s described in the pr eamble of claim 1 1 . 

f2 Recent developments in audio and video systems require the use of multiple channel 

s 1 processing and reproduction with acoustic echo cancellers (AEC) and noise cancellers. 

111 10 For example 4n mini group video conferencing systems, multiple channel transmission leads to 
~ a 

q a better "localization" of the diverse people in the room. This enhances the intelligibility and 

~ ; naturalness of the speech. / 

rkj?£ Further multiple channel echo cancellation is needed in voire controlled stereo audio and 

rv/K video equipment^such as television receivers, radio receivers, CD players^etc. 

15 A multiple channel AEC can k^-generST not be created by simple combination of multiple 
single channel AEC's. 



F rom the United Stat e s patent US A 5,828,756 a method and apparatus arc 
knoAArn of a stereophonic communication system 3uch as a teleconferencing system, which 



20 involves selectively reducing the correlation between the individual channel signals of the 
jj^s^ stereophonic system. Herein ; non-linearities are added to the input signals to reduce the 

correlation. However by adding these non-linearities, audible artifacts in the output signals are 

A 

introduced. These non-linearities can (sometimes) be accepted in teleconferencing systems but 
are certainly not acceptable in other applications^uch as supplying music, etc. 

It is, inter alia, an object of the invention to provide a filter, which overcomes 
•j^^ ($~J^ the-objecfions of the prior a rt. To this e n d a first aspect of the invention provides a fil l er as 
claimed in claim 1 . 
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Herewith^ the performance of the adaptive filters is improved without a huge increase in 
^_ computational complexity. 

A socond a s pect of the invention prov ides a ■ sig naUpEacessi&g^esric 8. 

5 A-thir d aspect of the invention provides-a-t^econferenGmg-system-as-daim^d in - claim 1 1 . 



claim~T2. 



(X--10 A- fi fth as p p ^t th^ in v ^nti^n provi4^-fHWs ^ ^ ^^ ^ c^ a im H in H aim ]_\_ 

— A^sixth-aspe ct uf the invention-pr ovides-a-Tnethed^^ 

ffi 

An ^rnhnHimpnt r>f tV|e invention comprises the features of claim 2. 



15 



The invention and additional features, which may be optionally used to 
implement the invention to advantage, will be apparent from and elucidated with references to 
the examples described below, hereinafter and shown in the figures . Herein shows: - 



0^ 

20 Figure 1 schematically^ the multiple input adaptive FIR filter, according to the 

A— invention^ I ^ 

Figure 2 schematically the calculation of the output of the FIR filter, according 
to the invention.) A 

Figure 3 schematically, the calculation of Y in the Multiple Input Partitioned 
25 Frequency Domain adaptive filter, according to the invention for the case with direct inverse 
power estimation*} 

Figure 4 schematically^the calculation of the coefficient vectors w i according 

to the invention^ J 

Figure 5 a schematic example of a stereo echo cancellation in a 
30 teleconferencing system according to the invention,^ 
^ Figure 6 ^m??recletailed schematic example of a teleconferencing system 

according to the invention- J J 
£k^/ Figure 7 a schematic example of a voice contra cted device according to the 

ft / invention,* and 




PH-NL000172 

■ . ' . • ' 3 29.03.2000 

r. ' ' St° w ^ 

Figure 8 a schematic example of a noise canceller according to the invention. 

In the description^the equations, matrices, ©teare shown as described below. 
Signals are denoted by lower case characters, constants by upper case. Underlining is used for 
5 vectors, lower case for time domain, and upper case for frequency domain. Matrices are 
denoted by bold face upper case, like- 1. The dimension is put in superscript (e.g.lhe BxQ 
matrix X is given by JY*'^, for a square matrix, the second dimension is omitted). Diagonal 

\ys ^ matrix -are denoted by a double underline, Kke P , with its diagonal denoted as P = diag Vr ( . 
a ** l>Vf ~ ~ 

fy^ A subscript /, like Wj denotes the f th version. The Ar'th element of w is given by (w)^ Finally, 
^ Andres 
jl^iO appendingj^fenotes the time index, (.) denotes the transpose, (.)* the-eomplexconjugate y 

and (.) h the Hermitain transpose (complex conjugate transpose). ^ 

A general multiple input adaptive FIR filter, depicted in-figwe-l , uses the S 
signals x 0 [k]^^il x s-l [k] to remove unwanted components correlated with these signals in 

l y 15 the signal e[k]. The signals x 0 [k] j^Hrx s _ x [k] are input to S FIR-filters W 0 until W s _ x , with 

^ „ -to ^ 

outputs e 0 [k] wittT e s _ x [k]. The goal of the update algorithm is to adapt the coefficients of the 
^ [J: L FIR filters in such a way that the correlation between r[k] and the input signals x 0 [k] and 
t; x s _ x [k] is removed. 



20 

For 5 > a > 0 the FIR filter W a performs the convolution of the signal x a [k] and 
the coefficients Wa, 0 [k] ... w^N.ifk] of that filter. The output signal e a [k] of such a filter can be 
described as follows 

25 eAk] = Z Xa [k-i] wJk]={x:[k]) (1) 

/=0 



with^ foi S ^ u L 0 
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(2) 



.oM 



V a 



(3) 



The output of the multiple input adaptive filter is given by 



s-i 



a=0 



(4) 



, These filter parts of the separate (adaptive) filters Wo until Ws-i can be 

ho implemented efficiently in frequency domain with help of partitioning, block processing and 
l Discrete Fourier Transforms (DFTs). A reduction in computational complexity is obtained 
( since the convolutions per sample in the time domain transform to elementwise multiplications 
per block in the frequency domain. We use block processing with block length B and DFTs of 
length M, with M>N + B-1. The transformation of the input signals can be described for 
15 S">a>0by 



X"[kB]=F M 



(x a [kB-M + \\\ 
x a [kB] 



(5) 



where F M is the M x M Fourier matrix. The (a ? b)'th element (for 0 < a < M,0 <b<M )of the 
20 Fourier matrix is given by 



- jlTiab 



M 



(6) 



where / = V-T 
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The filter can then be computed in the frequency domain, by 



in=(o i*)fixrNkr[*i?]= 



a=0 



(e[kB-B + l]\ 
e[/cB] 



(7) 



Note that the frequency domain filter coefficients are related to the time domain coefficients, 
for all S > a > 0 this can be denoted by 



W"[l] = F M 



w 



0 



(8) 



V ii J 

To obtain an efficient implementation, the block length B must be chosen in the 
same order as the filter length N, which results a large processing delay. 

To reduce the processing delay, the filter can be partitioned into smaller pieces 
of length B^and with g - [N I B~\ ( we get the implementation o f ifgu^c -2, that can be described 

by 



S-\ g m 



(9) 



a=0 i=l a 



15 with W_^M =vM 



w 



[/] 



w 



aJB+B-\ ['] 

0 



^ For the update part of the filteij one can use S separate update algorithms to 

improve convergence behavior the input signals can be decorrelated separately in the time 

domain by using RLS like algorithms, leading to a huge computational complexity. 

Complexity reduction can be obtained by implementation in the frequency domain with 

20 (partitioned) Block Frequency Domain Adaptive Filters, as described in G.P.M. Egelmeers, 

A 

Real time realization of large adaptive filters, Ph.D. thesis, Eindhoven University of 
Technology, Eindhoven (The Netherlands), Nov. 1995. When there is correlation between the 
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input signals of the filters^this might still lead to very bad convergence behavior, due to the 
non-uniqueness problem. 

In this application^ is proposed to use a partitioned algorithm in the frequency 
domain that reduces the effect of the^crosscorrelation-between the input signals on the 
algorithm's convergence behavior. To reduce complexity, block processing v with block length 
A to compute the sum of A consecutive updates with each iteration, is used. For S > a> 0,the 

coefficient vectors w„ [lA] are partitioned into g u parts of length Z with g u = [N I Z~| such that^ 

A 

for S > j > 0 



10 



1 »£oM . 



with for S > j>0 and g u >i>0 



7,/Z+Z-l 



15 



A Fourier transform length L is used with L > Z + A - 1 , we define the input signal Fourier 
transforms for S > a > 0 as 

K L a [lA] = ¥ L .x L a [lA] 



20 



(x a [lA-L + l]\ 
x a [lA-l] 



The diagonal matrices X L [lA] contain the vector X_ L a [lA] as main diagonal, so for S > a > 0 
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X L a [lA] = diag\x L a [lA]\. 



An overlap-save method is used to compute the correlation involved in the adaptation process 
5 in the frequency domain, the frequency domain transform of the residual signal vector equals 



R L [lA] = F L 



fQL-A > 



10 



The set of update equations for the filter coefficients in the MFDAF (Multiple Input 
Frequency Domain Adaptive Filter) algorithm can now be defined for g u > i > 0 by 



*£[(/+ 1)4 



15 



+ G 5 



Z.S.L 



(Y L s jA-iZ])* 

^[lA-iZp 
(Y L o [lA-iZ])* 



R L [IA] 



with 



= 2a(p Si [/^])" 



(x[-[/^])* 



20 
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t S.Z.S.L 



0 



0 



0 



Z,L 



Z.L 



qZ.L G Z.L 



5 with 



G Z,L = (jZ 0 Z,I-2^ F L)-> 



10 The input channel's power matrix P S L [lA] is defined by 



P s L [lA] = j- {x S L - L [lA])* .(X S L L [IA]) } 



15 where 



X Sii [/^] = 



20 



The expectation operator e{} of the above equation has to be replaced by an estimation 
routine. 
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The power matrix P s L \lA\can be estimated by 

Y SL [lA] = (\-y)P SL [(l -l)A] + ?-( X SL ' L [IA] )' (X SL - L [lA\> t (10) 

L 

To reduce the number of multiplications, the stepsize parameter a of equation is 
incorporated in the above power estimation routine by defining 

K L [lA] = j-V SL [lA] (11) 

SO 

2a(p Si [/4- 1 =(pf[/4-\ 02) 
Estimation of the power matrix L [lA\ can then be done by 



rr; p < f t N=(i-r)p a si [(/-^]+fV( ^ sll m n y. 03) 

21 2 ax, 

e Direct application of this algorithm leads to stability problems. When the input signal power in 

IS a certain frequency bin is very small, the power in that bin will decrease to a (very) small 

[U value. The inverse of the matrix will then have large values and will be inaccurate (due to 

p 15 numerical and estimation errors) In the ideal case, the eigenvalues of the power matrix 

l s 

estimate cancel the eigenvalues of the input signal power matrix. Due to estimation errors/his 
goal is only approximated, and the mismatch introduces a deviation from the ideal ^ 
convergence behavior and might even lead to instability. Especially when some of the 
eigenvalues of the estimate of the inverse power matrix get large, and do not (exactly) cancel a 

20 (small) eigenvalue of the input signal power matrix, instability might occur. A lower limit to 

the eigenvalues of the estimate of the power matrix can solve this problem. In the single 
j^, channel case (or when we forget the cross-terms^ we can solve this problem by applying a 
lower limit to the lower values. We can do this because the eigenvalues of a diagonal matrix 
are equal to the elements of the diagonal, so we actually limit the eigenvalues. In the multiple 
0^ 25 channel case, we also have to limit the eigenvalues to assure stability, but these no longer equal 
the elements on the diagonal. 

We know^however^that all eigenvalues of the power matrix are positive. We can 
now create a lower limit on the eigenvalues by shifting them by the suggested minimum. We 
know that for all eigenvalues X of a matrix A, the determinant of A + (P min — A,' ) • I must be 

30 zero. So for all X' , eigenvalue of A + P min I , there must be a X, eigenvalue of A, such that 
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/T =X+P m in (and the other way around). This means that by adding a constant P m i n to the main 
diagonal of a matrix, all eigenvalues of that matrix are shifted by P m in ). So we define: 



2a 



which results in 



j-j The effect of this shifting of the eigenvalues on the (theoretical ideal) convergence behaviour 

tML- io of the algorithm will be very small, and^n practice^e algorithm is much more stable. 



Although [f m [IA] is a sparse matrix, computing its inverse would still require L inversions of 



SxS matrices, which takes jkC the order of L -S operations. As we however do not need the 
15 inverse itself, but only its matrix- vector product with the input signal^ we can also look at it as 
O solving the system 



KLM 













x,' [ia]' 


V =o L J J 







20 

\y which requires jjf^the order of L-S 2 operations. Another option is to estimate the inverse of 
P S L [lA] directly, which also results in a number of operations proportional to LS 2 . 

^L^However, also in this case ,we have to limit the eigenvalues to assure stability. 
25 A 

A simple algorithm is given by: 
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(p 5 - L [LifF X =(\ + y) (p SL [(l-i)A]y i -^Q SLL [lA] (Q SL ' L [lA]Y 



with 



Q SLL [IA]=(V SL [IA])- 



X L [lA] 

= 5-1 L J 



£ M J 



= (p SL [lA]y.( X SLL [lA] y 



(14) 



10 We can incorporate a, which results in 



5 y 



20 



The above algorithm does not guarantee a matrix (p a 5 L [lA^j 1 with positive eigenvalues, and, 

therefore introduces a lot of stability problems. In the single channel case we are able to 
stabilize the algorithm by using a lower limit on the estimate, which automatically results in 
positive eigenvalues because the matrix is diagonal^ but that is not possible in the multiple 
channel case. 



Positive eigenvalues 



25 



An exact transformation of an algorithm for estimating L [lA] with positive eigenvalues will 

lead to an estimation algorithm for the inverse with positive eigenvalues. This can be done by 
using the matrix inversion lemma. When there is matrix A such that 

A=B + C D E (15) 



then the inverse matrix (A) of A can be expressed by 
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(A)' 1 =(B)~ l -(B)~ x C ((D)- 1 +E-(BT' -Cr 1 E-^BT 1 • (16) 

By choosing 
A=Pf'[£4] 

b = (l-r)-P" 

5 C=(X *■»[«])• (17) 

D =-21_ I' 
2aL 

E = C* =(X Si ' £ [/^] )' 



=i and 

^ Q = (B)" 1 c 

J5J - E (B)-' (18) 

o = ~ (P ^[(/-iMl)- 1 -(x Sii |M])* 



Q 10 we obtain by using equation (14) 



(P* L [lA})- 1 =— !— P a st [(/-l)^] _1 -Q s - t " t [L4]-(D i [i4])- 1 (Q Sii [L4]) / ' (19) 
1-7 — 



with 

15 

D L [U]=—I L +(X SLL [lA]y -Q SL - L [lA]- (20) 
- Y 



Algorithm (19) involves no matrix inversion, and only L/2 + l divisions, as the matrix 
D A [lA] of equation (20) is a real valued diagonal matrix. 

20 



Limits on the eigenvalues 
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An operation on the inverse power matrix that is equivalent to adding a constant to the 
diagonal of the (non-inverse) power matrix, would solve the problem. Adding a full 
(S * L) x (S • L) identity matrix and trying to find an equivalent operation on the inverse power 
matrix with the matrix inversion lemma^esi 
inversion we would like to avoid, so we try 



^/matrix with the matrix inversion lemma^esults in an algorithm that requires the matrix 



^t^]-0-r)-i;i[(/-iM]+/ F -(x£ 1 w ^]) , -(xL"[«]y 

2aL 



(21) 



with 



P10 



S15 



SL 



(22) 



As the matrix I has rank S • L and the product matrices (Xf^f 1 [lA])* • (X£ m L > L [lA])' and 
(X SL > L [lA])* ■ (X 5 ^ [lA])' both have a rank of (at most) lathis is not possible for S>1 . As we 
need the average of (X SL L [lA])' -(X S L - L [lA]y , we can find a solution by taking the average 
over S consecutive updates. We will try to find X^f' L [lA] such that 



\s~\ 



*{S ( x ^fc> + °W • (x^- £ [(/ + aM])«| 

= e\Y l (X SL - L [(l + a)A])' •(X Si - t [(/ + a)^])'} + P min -2aLS l SL 



Lo=o 



(23) 



20 



with for / - / mod S 











+ 











=x Si - i [w]+ 



(24) 
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u 



^ W 0,S-1 



For 5 = 1 we get 



"u "i.o 

W 0,l W 0.0 J 



(25) 



(26) 



For 5 > 1 there are an infinite number of solutions. If we try to keep the maximum distorsion 
V 

(the largest matrix element) as small as possible, we have to choose for all S > / > 0 and 



S > i > 0 



(27) 



If there is a real symmetric matrix U L for iS=Z, } then a real symmetric matrix U 2i for S = 2L is 
given by ^ 

tl ^ 

(28) 



U 2L = 



15 



Using the above equation^we can construct all U 2 with / > 0 . If 5 + 1 is not a power of two, 
then we will use the matrix U 2 ' where 2' > S + 1 > 2' 1 5 and use the last S rows. In table 1 fe e 



power matrix estimation algorithm using a direct inverse estimation with limits on the 
eigenvalues is summarized. 
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Initialization 

1 g _ 2^2 5+11 



for i-\to log 2 (S u )do 
2. begin 



u 2 ' = 



/^|j2'- 1 TT 2 



IT 



u 2 



u 



3. Initialize power matrix. 



Iteration 
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10 



= 5-1 



W S u -l,/modS u 



*S u -S+l,/modS w * 
W 5 u -5,/mod5 tf ^ J 



2. Q^M = -J-.(p*i [(/-1)4- .(XJ^M)- 

1-7 

3. D L [lA] = — I L +(X^' L [lA]y Q SLL [lA] 

- r 

4. Calculate (D L [lA]y l 

(pSLMr' =7 J -(p«;L[(/-im])- 1 

5. i-r 

- Q S LJ [IA\ (D L [Li])- 1 ■ (Q S L - L [lA]) h 

Table 1 : Direct inverse power update with limits. 



15 



Note that the inverse of P S L [lA] is also a sparse matrix with the same structure and we define 



cp si r 1 = 



T £ [W] ... T L [Ml T* [/^] 

= 1,5-1 L J =1,1 L J =1,0 L J 

, T L [lA] ... T 1 [Ml T l [lA] 

y —o,s-\ 1 J =o,i L J =o,o 1 J 
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where for 0 < / < S and 0 < / < S 



5 



a/ Figure 5, shows schematically an example of a teleconferencing system TS5 

using a stereo echo canceller SEC5 with adaptive filters AF5 (only one shown). The 
teleconferencing system comprises a far room FR5 and a near room NR5. The adaptive filter 
AF5 has to filter the stereo echo signals. 

q 10 Figure 6 shows an example of a stereo echo canceller SEC6 used in a 

teleconferencing system TS6. The stereo echo canceling has to be performed between the near 

CP room NR6 and the far room FR6. In this example^ also programmable filters PF61 and PF62 

* ; are used to improve the performance of echo canceling. Programmable filters are described in 

« ^^ A»4, 9 03,247 . 1 ' 

^ 15 Further also the output of the programmable filters is supplied to a dynamic 

O A- 

echo suppressor DES6, which is coupled with an output to the output of the stereo echo 



canceller. Dynamic echo suppressors are described in WO 97 45995 . 

p A full stereo commumpation requires four stereo AECs, two on the near end 

f^j side and two on the far end side. In^ figui e 6- only o n e of tho se, echo cancellers is depicted. Note 

|^ 20 that on each side^we can combine the input signal delay-lines, the FFTs and the multiplication 

by the inverse power matrix of the two echo cancellers, which implies that the relative extra 

computational complexity for removing the crosscorrelation is even further reduced. The 

0^ performance of the A£Cs^further improved by adding Dynamic Echo Suppressors as shown. 

Figure 7 shows another application wherein a stereo echo canceller SEC7 is 

0^ 25 used in a vpice - controlled audio (and video) system VCS7. To be able to recognize the local 

^ speaker by a voice recognition engine^we have to cancel the sound emitted by the audio set 

through the loudspeakers. This is done by using the stereo echo canceller SEC7. To improve 

i CUS0 , iTu tWi Zx~QJt*fle , 
^ the stereo echo canceling also in tnis - exampl e the programmable filters PF71 and PF72 are 

A. 

used, and the Dynamic Echo Suppresser DES7 is used. The output of the Dynamic Echo 
30 Suppressor is coupled to a voice recognizer VR7 for handling the filtered signal. 

Figure 8 shows an example of a noise canceller NC8 for canceling the noise 
received on microphones in a room R8 together with a speech signal spl from a person in the 
room. In this example, the microphones supply signals to a beam former BF8 wh tch beSm 



PH-NL000172 

■ ' . ■ ■ 17 29.03.2000 

£10^ rfemrcr supplies signals to the noise canceller NC8 and to programmable filters PF81, PF82 

and PF83. Furtheij the noise canceller comprises a dynamic echo suppressor DES8. The output 

of the Dynamic Echo Suppressor is coupled to the output of the noise suppressor to supply an 

estimate of the received speech sp2. 
{X^y Also in the multiple input noise canceller we can apply a DES (which is^in fact, 

A- ' 

not suppressing an echo, but is similar to the DES in the AEC's case) and programmable 
^ — filters to improve performance, as shown in £igwrer8. An extra problem is that the inputs of the 

filters may. contain some elements of the desired signal ("signal leakage"), because the 

. bea mfotHIci ' is not perfect. When the desired signal is speech signal, a speech detector can be 

1 0 used to improve the behaviour of the MFD AF. 

Q Above some examples of application of a stereo echo canceller and of a noise 

S v\jo\tz<L 

canceller are described. It is to be Jioticed -that the invention can be used in different 

iA- 

applications and is not restricted to the described applications. 



